Papers with deep neural networks
Fine-grained Interpretation and Causation Analysis in Deep NLP Models (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | Despite the proven efficacy of deep neural networks at-large, their opaqueness is a major cause of concern. |
| Approach: | They will present research work on interpreting fine-grained components of a neural network model from two perspectives, i) fine-grain interpretation, and ii) causation analysis. |
| Outcome: | This paper presents work on interpreting fine-grained components of a neural network model from two perspectives, i) fine-grain interpretation, and ii) causation analysis. |
CALOR-QUEST : generating a training corpus for Machine Reading Comprehension models from shallow semantic annotations (D19-58)
Copied to clipboard
Frederic Bechet, Cindy Aloui, Delphine Charlet, Geraldine Damnati, Johannes Heinecke, Alexis Nasr, Frederic Herledan
| Challenge: | Recent large corpora of triplets have opened the door to supervised machine learning approaches for Question-Answering. |
| Approach: | They propose to generate questions from the semantic Frame analysis of large corpora using a CALOR-QUEST resource in French and use it to improve machine reading comprehension. |
| Outcome: | The proposed method generates questions from the semantic Frame analysis of large corpora and then tests them on the CALOR-QUEST resource in French. |
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
UKP-SQuARE v2: Explainability and Adversarial Attacks for Trustworthy QA (2022.aacl-demo)
Copied to clipboard
Rachneet Sachdeva, Haritz Puerto, Tim Baumgärtner, Sewin Tariverdian, Hao Zhang, Kexin Wang, Hossain Shaikh Saadi, Leonardo F. R. Ribeiro, Iryna Gurevych
| Challenge: | Question Answering (QA) systems rely on deep neural networks, which are difficult to interpret by humans. |
| Approach: | They propose an interpretable model that provides an explanation infrastructure for comparing models based on saliency maps and graph-based explanations. |
| Outcome: | The proposed methods can be used to compare models based on saliency maps and graph-based explanations. |
Deep Bayesian Learning and Understanding (C18-3)
Copied to clipboard
| Challenge: | COLING 2018 is a conference for researchers and practitioners working on machine learning and deep learning. |
| Approach: | a tutorial on machine learning and deep learning will be presented at COLING 2018 . the tutorial will focus on statistical models, deep neural networks, sequential learning and natural language understanding . |
| Outcome: | This tutorial will present the latest advances in deep Bayesian and sequential learning at COLING 2018 . |
Fast Word Predictor for On-Device Application (2020.coling-demos)
Copied to clipboard
| Challenge: | a proposed word prediction model is developed for a chat application serving more than 100 million users. |
| Approach: | They propose a fast word predictor that reduces memory size and inference time on mobile devices. |
| Outcome: | The proposed model reduces memory size and inference time on a mobile device compared with a standard neural network . it achieves robust performance by learning on large text corpora and is available on microsoft's chat app . |
Long Document Summarization in a Low Resource Setting using Pretrained Language Models (2021.acl-srw)
Copied to clipboard
Ahsaas Bajaj, Pavitra Dangati, Kalpesh Krishna, Pradhiksha Ashok Kumar, Rheeya Uppaal, Bradford Windsor, Eliot Brenner, Dominic Dotterrer, Rajarshi Das, Andrew McCallum
| Challenge: | Existing abstractive summarization methods only achieve 17.9 ROUGE-L in low-resource settings. |
| Approach: | They propose to use a modern abstractive summarization algorithm to extract salient sentences from long documents to improve their performance. |
| Outcome: | The proposed method beats several competitive salience detection baselines and the identified salient sentences agree with independent human labeling by domain experts. |
A Neural Pipeline Approach for the PharmaCoNER Shared Task using Contextual Exhaustive Models (D19-57)
Copied to clipboard
| Challenge: | NER and concept indexing perform named entity recognition and concept identifiers (CUIs) in a knowledge base. |
| Approach: | They propose a neural pipeline approach that performs named entity recognition (NER) and concept indexing (CI) they use bi-LSTM to capture the semantic information of a sequence and classify them into entities or no entities . |
| Outcome: | The proposed approach performs named entity recognition (NER) and concept indexing (CI) which links them to concept unique identifiers (CUIs) in a knowledge base. |
SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models have limitations to generalize to diverse semantic phenomena, and it is unclear whether they can capture compositional meanings. |
| Approach: | They propose a systematic generalization testbed based on Natural language semantics to map natural language sentences to multiple meaning representations. |
| Outcome: | The proposed model can generalize to unseen combinations of quantifiers, negations, and modifiers, but not to the others. |
NxPlain: A Web-based Tool for Discovery of Latent Concepts (2023.eacl-demo)
Copied to clipboard
| Challenge: | Interpretability of deep neural networks has gained a lot of attention in recent years, especially in NLP, where state-of-the-art models are being widely deployed and used in practice. |
| Approach: | They propose to analyze what linguistic and non-linguistic knowledge is learned within deep neural networks and highlight the salient parts of the input. |
| Outcome: | The proposed tool is useful for debugging, unraveling model bias, and for highlighting spurious correlations in a model. |
On the Calibration and Uncertainty of Neural Learning to Rank Models for Conversational Search (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to rank documents in decreasing order of their probability of relevance are not well calibrated and have several sources of uncertainty. |
| Approach: | They propose to calibrate deterministic neural rankers for conversational search problems . they then use two techniques to model the uncertainty of neural ranker's uncertainty . |
| Outcome: | The proposed rankers output a predictive distribution of relevance as opposed to point estimates. |
Familiar words but strange voices: Modelling the influence of speech variability on word recognition (2021.eacl-srw)
Copied to clipboard
| Challenge: | Despite the lack of acoustic-phonetic invariance in speech, listeners can reliably recognize spoken words despite the lack aural-phonemic invariancy. |
| Approach: | They propose a deep neural model which is trained to retrieve the meaning of a word given its spoken form, a task which resembles that faced by a human listener. |
| Outcome: | The proposed model is more sensitive to dialectical variation than gender variation and more related to related languages. |
Evaluating Lottery Tickets Under Distributional Shifts (D19-61)
Copied to clipboard
| Challenge: | Recent research suggests deep neural networks are dramatically over-parametrized. |
| Approach: | They propose that large, over-parameterized neural networks consist of small, sparse subnetworks that can be trained in isolation to reach a similar (or better) test accuracy. |
| Outcome: | The proposed models can achieve commensurate performance using the same initialization as the original model. |
Model-agnostic Methods for Text Classification with Inherent Noise (2020.coling-industry)
Copied to clipboard
| Challenge: | Text classification is a fundamental problem in natural language processing, but its performance relies on high-quality annotations. |
| Approach: | They propose to use model-agnostic methods to handle inherent noise in large scale text classification that can be easily incorporated into existing machine learning workflows with minimal interruption. |
| Outcome: | The proposed method outperforms baselines by up to 10% in classification accuracy while requiring no network modifications. |
Large Scale Author Obfuscation Using Siamese Variational Auto-Encoder: The SiamAO System (2020.starsem-1)
Copied to clipboard
| Challenge: | Existing approaches to author obfuscation are largely heuristic, but they can be used to attack author identification. |
| Approach: | They propose a deep learning architecture for constructing adversarial examples against similarity-based learners and explore its application to author obfuscation. |
| Outcome: | The proposed architectures show that they can be used to attack author obfuscation . the proposed architecture shows that it can be applied to obliquacy of text . |
Emotion Classification in a Resource Constrained Language Using Transformer-based Approach (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to classify Bengali text into six basic emotions are infancy for resource-constrained languages like English, Arabic, Chinese and French. |
| Approach: | They propose a transformer-based technique to classify Bengali text into one of the six basic emotions: anger, fear, disgust, sadness, joy, and surprise. |
| Outcome: | The proposed technique outperforms all other techniques by achieving highest weighted f_1-score on the test data. |
When and Why a Model Fails? A Human-in-the-loop Error Detection Framework for Sentiment Analysis (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods for sentiment analysis are difficult to assess for erroneous predictions that might exist prior to deployment. |
| Approach: | They propose a framework for error detection based on explainable features that can detect erroneous model predictions on unseen data with high precision. |
| Outcome: | The proposed framework detects erroneous model predictions on unseen data with high precision, given limited human-in-the-loop intervention, and can be deployed on unselected data with a high accuracy. |
Improving Relation Extraction with Knowledge-attention (D19-1)
Copied to clipboard
| Challenge: | Existing attention mechanisms are data-driven, but most are data driven. |
| Approach: | They propose a knowledge-attention encoder which integrates prior knowledge from external lexical resources into deep neural networks for relation extraction task. |
| Outcome: | The proposed system outperforms existing CNN, RNN, and self-attention based models on a large-scale relation extraction dataset. |
A Flexible, Efficient and Accurate Framework for Community Question Answering Pipelines (P18-4)
Copied to clipboard
| Challenge: | Community Question Answering is a research area that benefits from deep linguistic analysis . previous cQA challenges have shown that neural approaches are not enough to deliver state-of-the-art results . |
| Approach: | They propose a framework to distribute computation of cQA tasks over computer clusters . community question answering is a research area that benefits from deep linguistic analysis . |
| Outcome: | The proposed framework scales to large datasets and delivers fast processing. |
Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement (P18-1)
Copied to clipboard
| Challenge: | a number of post hoc explanation methods for deep neural networks have been proposed . due to the complexity of the DNNs they explain, these methods are necessarily approximations and come with their own sources of error. |
| Approach: | They propose two evaluation paradigms that cover two important classes of NLP problems . they propose LIMSSE, LRP and DeepLIFT as the most effective explanation methods . |
| Outcome: | The proposed methods are most effective for explaining deep neural networks in NLP . the proposed methods can explain complex models without manual annotation . |
CASN:Class-Aware Score Network for Textual Adversarial Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to counteract adversarial attacks can be divided into two directions, adversarials defense and adversarially detection. |
| Approach: | They propose a score-based generative method to implicitly model the data distribution using a log-density distribution and supervised contrastive learning to guide the estimation using label information. |
| Outcome: | The proposed method improves on three text classification tasks on four advanced attack algorithms. |
Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances apply artificial intelligence to predict clinical events or infer the probable diagnosis for clinical decision support. |
| Approach: | They propose a hypernetwork-based approach that generates task-conditioned parameters and coefficients of multitask prediction heads to learn task-specific prediction and balance the multitask learning. |
| Outcome: | Experiments on clinical notes from the real-world MIMIC database show that the proposed model can achieve better performance than baselines and improve zero-shot prediction on unseen diagnoses. |
Cut to the Chase: A Context Zoom-in Network for Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Recent deep-learning based models suffer from reasoning over long documents and do not trivially generalize to cases where the answer is not present as a span. |
| Approach: | They propose a novel context zoom-in network (ConZNet) that can skip through irrelevant parts of a document and generate an answer using only the relevant regions of text. |
| Outcome: | The proposed architecture outperforms state-of-the-art results by 12.62% (ROUGE-L) relative improvement on the recently proposed and challenging RC dataset ‘NarrativeQA’. |
DropMix: A Textual Data Augmentation Combining Dropout with Mixup (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to overcome overfitting in text learning do not consider dimensionality . dimensionalization is important for deep neural networks to overcome the problem . |
| Approach: | They propose a saliency map-based approach to overcome overfitting in text learning . they propose augmentation regularization methods such as Dropout and Mixup to improve regularization . |
| Outcome: | Empirical results show that the proposed approach overcomes overfitting in text learning . dropout and mixup methods are effective in enhancing regularization . |
Enhancing Job Evaluation with Data Augmentation and Text Classification (2026.acl-industry)
Copied to clipboard
| Challenge: | Recruiters rely on job titles, role descriptions, and responsibility levels to determine job grades and salary structures. |
| Approach: | They propose to semi-automate job evaluation by fine-tuning a RoBERTa model for classification and using Gemini to generate synthetic job descriptions for rare job titles. |
| Outcome: | The proposed method improves job evaluation by boosting consistency and speeding up workflows. |
Deep Neural Networks for Coreference Resolution for Polish (L18-1)
Copied to clipboard
| Challenge: | Existing deep neural networks for coreference resolution for Polish have been used to resolve textual fragments that refer to the same entity in the discourse world. |
| Approach: | They propose a system combining the best deep neural architecture and sieve-based coreference resolvers ordered from most to least precise to achieve the highest results. |
| Outcome: | The proposed system improves the state of the art for Polish by 0.53 F1 points, reaching 81.23 points of the CoNLL metric. |
Sanaphor++: Combining Deep Neural Networks with Semantics for Coreference Resolution (L18-1)
Copied to clipboard
Julien Plu, Roman Prokofyev, Alberto Tonon, Philippe Cudré-Mauroux, Djellel Eddine Difallah, Raphaël Troncy, Giuseppe Rizzo
| Challenge: | Coreference resolution is a challenging task in Natural Language Processing . since a few years, the biggest step forward has been made using deep neural networks . |
| Approach: | They propose to improve coreference resolution by adding semantic features to a top-level deep neural network system . they evaluate a shared task dataset and compare it to the state-of-the-art system based on Stanford deep-coref . |
| Outcome: | The proposed system achieves 1.13% gain over the CoNLL 2012 dataset and the state-of-the-art system. |
Self-Training with Weak Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | State-of-the-art deep neural networks require large amounts of labeled training data that is expensive to obtain or not available for many tasks. |
| Approach: | They propose a weak supervision framework that leverages all available data for a given task . they leverage task-specific unlabeled data through self-training with a model that predicts pseudo-labels for instances that may not be covered by weak rules . |
| Outcome: | The proposed framework improves on state-of-the-art datasets on six benchmark tasks. |
“Yes, My LoRD.” Guiding Language Model Extraction with Locality Reinforced Distillation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for model extraction attacks on large language models are inadequate . existing methods neglect the inconsistency between training tasks and LLM alignment . |
| Approach: | They propose a model extraction algorithm that uses a policy-gradient-style training task to guide the crafting of preference for the local model. |
| Outcome: | The proposed algorithm reduces query complexity while mitigating watermark protection . it can extract various state-of-the-art commercial LLMs while minimizing query complexity . |
Meta Self-Refinement for Robust Learning with Weak Supervision (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent methods leverage self-training to build noise-resistant models . however, the teacher trained under weak supervision may have fitted a substantial amount of noise and therefore produce incorrect pseudo-labels. |
| Approach: | They propose a framework that encourages teacher to refine its pseudo-labels to effectively combat label noise from weak supervision. |
| Outcome: | The proposed framework outperforms state-of-the-art methods by 11.4% in accuracy and 9.26% in F1 score on eight NLP benchmarks. |
Exploring Transitivity in Neural NLI Models through Veridicality (2021.eacl-main)
Copied to clipboard
| Challenge: | Despite recent success of deep neural networks in natural language processing, the extent to which they can demonstrate human-like generalization capacities remains unclear. |
| Approach: | They propose an analysis method to evaluate whether models can draw inferences composed of veridical inference and arbitrary inference types. |
| Outcome: | The proposed model performs poorly on transitivity inference tasks, suggesting it lacks generalization capacity for drawing composite inferences from training examples. |
Neural Attention-Aware Hierarchical Topic Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural topic models (NTMs) use deep neural networks to learn topic information. |
| Approach: | They propose a variational autoencoder model that reconstructs sentence and document word counts using bag-of-words embeddings and pre-trained semantic embedders. |
| Outcome: | The proposed model lowers reconstruction errors at sentence and document levels and finds more coherent topics from real-world datasets. |
Towards Improving Adversarial Training of NLP Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods for generating NLP adversarial examples involve combinatorial search and expensive sentence encoders for constraining the generated instances. |
| Approach: | They propose to use vanilla adversarial training to train NLP models using a word substitution attack optimized for vanilla adversary training. |
| Outcome: | The proposed approach improves model performance and standard accuracy and can defend against other types of word substitution attacks. |
When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing DA methods naively add a certain number of augmented samples without considering the quality and the added computational cost of these samples. |
| Approach: | They propose a data-augmented DA technique that generates or reweights augmented samples . they say it is faster to train and can be plugged into any DA method . |
| Outcome: | The proposed technique is faster to train and more efficient than existing methods. |
Working Memory Networks: Augmenting Memory Networks with a Relational Reasoning Module (P18-1)
Copied to clipboard
| Challenge: | Recent advances in deep neural networks have enabled complex reasoning tasks. |
| Approach: | They propose a MemNN architecture with a working memory storage and reasoning module that retains relational reasoning abilities of relation networks while reducing computational complexity. |
| Outcome: | The proposed model retains the relational reasoning abilities of the RN while reducing its computational complexity from quadratic to linear. |
Domain Adaptation with Adversarial Training and Graph Embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing models for deep neural networks can handle data distributions between source and target domains, but they must deal with data distribution drifts. |
| Approach: | They propose a model that leverages unlabeled and labeled data from a related domain to deal with distribution drifts. |
| Outcome: | The proposed model improves over baselines on two real-world disaster datasets. |
Searching for Effective Neural Extractive Summarization: What Works and What’s Next (P19-1)
Copied to clipboard
| Challenge: | Recent years have seen success in the use of deep neural networks on text summarization, but there is no clear understanding of why they perform so well or how they might be improved. |
| Approach: | They propose to use different types of model architectures to improve extractive summarization systems. |
| Outcome: | The proposed framework achieves state-of-the-art on CNN/DailyMail by a large margin based on observations and analysis. |
Learning Efficient Task-Specific Meta-Embeddings with Word Prisms (2020.coling-main)
Copied to clipboard
| Challenge: | Word embeddings possess different lexical properties depending on the notion of context defined at training time. |
| Approach: | They introduce a meta-embedding method that learns to combine source embeddings according to the task at hand. |
| Outcome: | The proposed method improves performance on six extrinsic evaluations over other methods. |
All Information is Valuable: Question Matching over Full Information Transmission Network (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for question matching only transmit one kind of information while failing to utilize both kinds of information simultaneously. |
| Approach: | They propose a question matching network that can transmit both representation and interactive information together in a simultaneous fashion. |
| Outcome: | The proposed approach outperforms strong baseline models on two standard benchmarks. |
A Novel Framework Based on Medical Concept Driven Attention for Explainable Medical Code Prediction via External Knowledge (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict medical codes from clinical notes lack interpretability due to lengthy and noisy clinical notes. |
| Approach: | They propose a framework based on medical concept driven attention to integrate external knowledge for explainable medical code prediction from clinical notes. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on a benchmark dataset showing that it is more accurate than existing methods. |
Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention (2021.acl-long)
Copied to clipboard
| Challenge: | Generally, documents are truncated before being inputs to deep neural networks, resulting in missing keyphrases . evaluators use layer-wise coverage attention to cover all the critical points in a document . |
| Approach: | They propose a neural keyphrase generation model that identifies the salient sentences in a document and an extractor-generator that jointly extracts and generates keyphrases from the selected sentences. |
| Outcome: | The proposed model outperforms the state-of-the-art keyphrase generation methods on keyphrases generated from scientific and web documents. |
SAFR: Neuron Redistribution for Interpretability (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies on controlling neuron distribution for interpretability have focused on focusing on monosemanticity instead of focusing solely on feature interactions. |
| Approach: | They propose a method to regularize feature superposition by encoding representations of multiple features within a single neuron. |
| Outcome: | The proposed method improves model interpretability without compromising prediction performance. |
Dense Information Flow for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Recent advances in deep neural networks have improved learning performance for NMT . Residual connections allow features from previous layers to be accumulated to the next layer easily. |
| Approach: | They propose a densely connected NMT architecture that can train more efficiently for NMT. |
| Outcome: | The proposed architecture improves learning performance and attention quality on multiple datasets. |
A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models (2024.lrec-main)
Copied to clipboard
Chenyang Lyu, Zefeng Du, Jitao Xu, Yitao Duan, Minghao Wu, Teresa Lynn, Alham Fikri Aji, Derek F. Wong, Longyue Wang
| Challenge: | Large Language Models (LLMs) are introducing a new phase in machine translation . despite advances in MT, there are still many challenges to overcome . |
| Approach: | They propose to highlight several new directions for MT that are influenced by Large Language Models like GPT-4 and ChatGPT. |
| Outcome: | The proposed models offer vast linguistic understandings and bring innovative methodologies, such as prompt-based techniques, that have the potential to further elevate MT. |
Do explanations make VQA models more predictable to a human? (D18-1)
Copied to clipboard
| Challenge: | Existing explanations of a model's behavior are not used in interactive tasks like Visual Question Answering (VQA). |
| Approach: | They analyze existing explanations and their role in making a VQA model more predictable to a human by using human-in-the-loop approaches that treat the model as a black-box. |
| Outcome: | The proposed explanations make a model more predictable to humans, whereas human-in-the-loop approaches treat it as a black-box do. |
Unsupervised Information Extraction: Regularizing Discriminative Approaches with Relation Distribution Losses (P19-1)
Copied to clipboard
| Challenge: | Existing unsupervised relation extraction models are either generative or discriminative . however, they are hard to train without supervision and are unstable . |
| Approach: | They propose a skewness loss and distribution distance loss to improve the performance of discriminative based models. |
| Outcome: | The proposed models surpass current state-of-the-art on three different datasets. |
Beyond [CLS] through Ranking by Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on generative ranking models for Information Retrieval has focused on discriminative methods that learn a similarity function to compare questions and candidates answers. |
| Approach: | They propose to use a language model to train a ranking function that model the semantic similarity of documents and queries instead of discriminative ranking functions. |
| Outcome: | The proposed approaches are as effective as state-of-the-art discriminative models for the answer selection task and show unlikelihood losses are reduced for IR. |
ASEM: Enhancing Empathy in Chatbot through Attention-based Sentiment and Emotion Modeling (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models lack feature representations that capture the deep semantics of language and sensitivity to minor input variations, resulting in significant changes in the generated text. |
| Approach: | They propose an end-to-end model architecture called ASEM that performs emotion analysis on top of sentiment analysis for open-domain chatbots. |
| Outcome: | The proposed model outperforms existing models for generating empathetic embeddings, providing e-mpathetic and diverse responses. |
Characterizing the Impacts of Instances on Robustness (2023.findings-acl)
Copied to clipboard
Rui Zheng, Zhiheng Xi, Qin Liu, Wenbin Lai, Tao Gui, Qi Zhang, Xuanjing Huang, Jin Ma, Ying Shan, Weifeng Ge
| Challenge: | Existing defense approaches focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances. |
| Approach: | They propose a method that can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training. |
| Outcome: | The proposed method can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training. |
Fine-Grained Emotion Detection in Health-Related Online Posts (D18-1)
Copied to clipboard
| Challenge: | Emotion detection from health-related posts is based on a health-specific vocabulary that people use in OHCs. |
| Approach: | They propose to use deep neural networks and lexicon-based features to detect emotions in health-related posts. |
| Outcome: | The proposed method uses high-level and abstract features derived from deep neural networks combined with lexicon-based features to detect emotions. |
Combining Feature and Instance Attribution to Detect Artifacts (2022.findings-acl)
Copied to clipboard
| Challenge: | In this paper, we evaluate use of different attribution methods for aiding identification of training data artifacts. |
| Approach: | They propose hybrid methods that combine saliency maps and instance attribution methods to aid in identifying training data artifacts. |
| Outcome: | The proposed methods can be used to efficiently uncover artifacts in training data when a challenging validation set is available. |
RMLM: A Flexible Defense Framework for Proactively Mitigating Word-level Adversarial Attacks (2023.acl-long)
Copied to clipboard
| Challenge: | Existing defenses focus on improving robustness of the victim model in training, but neglect to mitigate adversarial attacks during inference. |
| Approach: | They propose a framework that confuses attackers and corrects adversarial contexts . their framework helps improve the robustness of the victim model during inference . |
| Outcome: | The proposed framework improves the robustness of the victim model in training . it also corrects abnormal contexts in the representation level and filtering out examples . |
A Neural-Symbolic Approach to Natural Language Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have enabled deep neural networks to perform natural language understanding tasks, but their performance can drastically deteriorate when logical reasoning is needed. |
| Approach: | They propose a framework for NLU based on analogical reasoning based upon neural processing and logical reasoning using both neural and symbolic processing. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two NLU tasks, question answering (QA) and natural language inference (NLI). |
Locally Aggregated Feature Attribution on Natural Language Model Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | a growing popularity of deep-learning models makes model understanding more important . feature attribution methods have shown promising results in computer vision but are not trivial . |
| Approach: | They propose a gradient-based feature attribution method that smooths gradients by aggregating similar reference texts derived from language model embeddings. |
| Outcome: | The proposed method outperforms existing methods on public datasets and key words detection tasks. |
Are Factuality Checkers Reliable? Adversarial Meta-evaluation of Factuality in Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Despite the progress of factual evaluation methods, they are limited in their opacity and lack the ability to assess the factuality of the summaries. |
| Approach: | They propose to use a meta-evaluation methodology to diagnose the fine-grained strengths and weaknesses of 6 existing top-performing metrics over 24 diagnostic test datasets. |
| Outcome: | The proposed method diagnoses the strengths and weaknesses of 6 existing top-performing metrics over 24 diagnostic test datasets and searches for directions for further improvement by data augmentation. |
A Survey on Recent Advances in Named Entity Recognition from Deep Learning models (C18-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc. |
| Approach: | They propose to use recurrent neural networks to generate NERs over characters, sub-words and/or word embeddings to improve named entity recognition. |
| Outcome: | The proposed architectures are better than those based on feature engineering and other supervised or semi-supervised learning algorithms. |
NeuralREG: An end-to-end approach to referring expression generation (P18-1)
Copied to clipboard
| Challenge: | Referring Expression Generation models typically rely on features such as salience and grammatical function to make decisions about form and content. |
| Approach: | They propose a new approach that makes decisions about form and content in one go . they use a delexicalized version of the WebNLG corpus to test the approach . |
| Outcome: | The proposed approach significantly improves over two strong baselines. |
Lessons from Natural Language Inference in the Clinical Domain (D18-1)
Copied to clipboard
| Challenge: | State of the art models with deep neural networks lack generalization capabilities in specialized domains where training data is limited. |
| Approach: | They propose a dataset annotated by doctors performing a natural language inference task grounded in the medical history of patients. |
| Outcome: | The proposed model outperforms existing models in the clinical domain by incorporating domain knowledge from external data and lexical sources. |
MA-BERT: Learning Representation by Incorporating Multi-Attribute Knowledge in Transformers (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for incorporating external attribute knowledge into deep neural networks are concatenating multiple attributes to word/text representation or treating them as biases to adjust attention distribution. |
| Approach: | They propose a multi-attribute BERT to incorporate external attribute knowledge into deep neural networks. |
| Outcome: | The proposed method outperforms existing models and models on three benchmark datasets. |
Annealing Knowledge Distillation (2021.eacl-main)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) is a powerful model compression technique for deep neural networks. |
| Approach: | They propose a method to feed the rich information provided by teacher’s soft-targets incrementally and more efficiently by annealing the teacher output incrementally. |
| Outcome: | The proposed method can be used on image classification and NLP language inference tasks with BERT-based models on the GLUE benchmark. |
“I Know Who You Are”: Character-Based Features for Conversational Humor Recognition in Chinese (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has focused on how to recognize punchlines from dialogues, but has neglected character information. |
| Approach: | They propose a character-fusion conversational humor recognition model that uses character information to recognize punchlines from dialogue. |
| Outcome: | The proposed model improves performance on Chinese sitcoms corpus and punchline identification. |
A Simple Model for Distantly Supervised Relation Extraction (2022.coling-1)
Copied to clipboard
| Challenge: | Recent methods focus on exploiting bag representations with complex de-noising scheme to achieve remarkable performance. |
| Approach: | They propose a BERT-based Graph convolutional network model that exploits bag representations . their model extracts key information from each instance and constructs a bag graph . |
| Outcome: | The proposed model improves on two benchmark datasets, i.e., NYT10 and GDS. |
A Batch Normalized Inference Network Keeps the KL Vanishing Away (2020.acl-main)
Copied to clipboard
| Challenge: | Variational Autoencoder (VAE) is widely used to approximate a model’s posterior on latent variables. |
| Approach: | They propose to let the Kullback–Leibler divergence individual follow a distribution across the whole dataset and analyze that it is sufficient to prevent posterior collapse by keeping the expectation of the KL’s distribution positive. |
| Outcome: | The proposed approach can avoid posterior collapse effectively and efficiently without introducing any new model component or modifying the objective. |
TableBank: Table Benchmark for Image-based Table Detection and Recognition (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing techniques for table detection and recognition are limited to document types and layouts. |
| Approach: | They propose to build a table detection and recognition dataset with weak supervision from Word and Latex documents on the internet. |
| Outcome: | The proposed dataset contains 417K high quality labeled tables and is publicly available. |
Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to label large-scale data are inadequate for distantly supervised relation extraction (DS-RE). |
| Approach: | They propose a multi-level structured (2-D matrix) self-attention mechanism for DS-RE using bidirectional recurrent neural networks. |
| Outcome: | The proposed framework significantly outperforms baselines on two publicly available DS-RE datasets in terms of PR curves, P@N and F1 measures. |
Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Tasks (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for uncertainty estimation are inadequate for safety-critical applications. |
| Approach: | They propose a method that uses the distances from neighbors and the ratio of labels in neighbors to estimate uncertainty. |
| Outcome: | The proposed method outperforms baseline and density-based methods in calibration and uncertainty metrics. |
How emotional are you? Neural Architectures for Emotion Intensity Prediction in Microblogs (C18-1)
Copied to clipboard
| Challenge: | Social media based micro-blogging sites like Twitter are used for expressing emotions and opinions. |
| Approach: | They propose to combine convolutional and fully connected layers in a non-sequential manner to train deep multi-task learning models trained for all emotions at once in unified architecture. |
| Outcome: | The proposed model outperforms the previous system by 0.044 or 4.4% on the WASSA’17 EmoInt shared task dataset. |
Topological Sort for Sentence Ordering (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work on sentence ordering task has framed it as a sequence prediction problem. |
| Approach: | They propose a new constraint solving problem and propose 'human evaluation' they propose to capture coherence in documents by arranging sentences in the correct order . |
| Outcome: | The proposed technique captures coherence in documents better than previous approaches. |
ATFormer: A Learned Performance Model with Transfer Learning Across Devices for Deep Learning Tensor Programs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Compilation-based methods with performance models have poor measurement accuracy and transferability between platforms. |
| Approach: | They propose a compiler that automatically generates tensors and automatically tunes them for different hardware platforms. |
| Outcome: | The proposed model reduces inference time and costs on modern DNN benchmarks. |
Interpretation of NLP models through input marginalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to interpret NLP predictions replace each token with a predefined value, resulting in misleading interpretations. |
| Approach: | They propose to marginalize each token out of the training data distribution to demystify the "black box" property of deep neural networks for natural language processing. |
| Outcome: | The proposed method marginalizes each token out of the training data distribution. |
Analytic Automated Essay Scoring Based on Deep Neural Networks Integrating Multidimensional Item Response Theory (2022.coling-1)
Copied to clipboard
| Challenge: | Essay exams have two drawbacks in that grading them is expensive and raises questions about fairness. |
| Approach: | They propose to use a multidimensional item response theory model to improve interpretability while maintaining scoring accuracy. |
| Outcome: | The proposed model improves interpretability while maintaining accuracy while preserving cost and accuracy. |
IDEAW: Robust Neural Audio Watermarking with Invertible Dual-Embedding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Traditional methods for embedding watermarks into audio have low capacity and unsatisfactory imperceptibility. |
| Approach: | They propose a dual-embedding wa- termarking model for efficient locating and a model that can withstand attacks. |
| Outcome: | The proposed model can withstand attacks with higher capacity and more efficient locating ability compared to existing methods. |
TextVerifier: Robustness Verification for Textual Classifiers with Certifiable Guarantees (2023.findings-acl)
Copied to clipboard
| Challenge: | a textual classifier must withstand word-level alteration attacks due to inherent vulnerability. |
| Approach: | They propose a formal verification framework with certifiable guarantees on deep neural networks in natural language processing against word-level alteration attacks. |
| Outcome: | The proposed framework provides an approximation of the maximal safe radius with tight bounds . it yields an efficient speed edge and reliable anytime estimation . |
Interactive Key-Value Memory-augmented Attention for Image Paragraph Captioning (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to image paragraph captioning ignore the past alignment information, resulting in repetitive captioning and incomplete captioning. |
| Approach: | They propose an Interactive key-value Memory-augmented Attention model for image paragraph captioning to keep track of attention history along with update-chain of decoder state. |
| Outcome: | Extensive experiments on a benchmark dataset demonstrate the effectiveness of the proposed model. |
The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Extending state-of-the-art language models to low-resource languages requires addressing what we call the low-Resource double bind. |
| Approach: | They propose a low-resource double bind to refer to the co-occurrence of data limitations and compute resource constraints. |
| Outcome: | The proposed model improves performance on frequent sentences but disparates on infrequent ones. |
Hyperbolic Capsule Networks for Multi-Label Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for classification of labels are limited by feature aggregation and encoding. |
| Approach: | They propose to use hyperbolic capsule networks to capture fine-grained label information . they also propose a new routing method to adaptively adjust capsule number during routing . |
| Outcome: | The proposed method significantly improves the performance of multi-label classification on tail labels. |
Asymmetric feature interaction for interpreting model predictions (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior work on feature interaction attribution studies focus on asymmetric interaction that only explains the additional influence of a set of words in combination, which fails to capture asymmetry influence that contributes to model prediction. |
| Approach: | They propose an asymmetric feature interaction attribution explanation model that explores asymmetry higher-order feature interactions in the inference of deep neural NLP models. |
| Outcome: | The proposed model outperforms state-of-the-art models on two sentiment classification datasets. |
Towards Interpretable Clinical Diagnosis with Bayesian Network Ensembles Stacked on Entity-Aware CNNs (2020.acl-main)
Copied to clipboard
| Challenge: | a novel framework for text-based diagnosis of diseases requires appropriate balance between accuracy and interpretability. |
| Approach: | They propose a framework that stacks Bayesian Network Ensembles on top of CNN to build an accurate yet interpretable diagnosis system. |
| Outcome: | The proposed framework outperforms the previous automatic diagnosis methods in accuracy performance and the diagnosis explanation of the framework is reasonable. |
Relation Classification Using Segment-Level Attention-based CNN and Dependency-based RNN (N19-1)
Copied to clipboard
| Challenge: | Recent work on relation classification has gained much success by exploiting deep neural networks. |
| Approach: | They propose a relation classification model using Segment-level Attention-based Convolutional Neural Networks and Dependency-based Recurrent Neural networks. |
| Outcome: | The proposed model is comparable to the state-of-the-art without external lexical features on the SemEval-2010 dataset. |
SenticVec: Toward Robust and Human-Centric Neurosymbolic Sentiment Analysis (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to tackle learning challenges such as knowledge forgetting and extensive computing resources are not effective. |
| Approach: | They propose a novel neurosymbolic method for sentiment analysis that places emphasis on human subjectivity within varying domain annotations. |
| Outcome: | The proposed method is lightweight, robust across domains and languages, efficient few-shot training, and rapid convergence. |
On Robustness of Neural Semantic Parsers (2021.eacl-main)
Copied to clipboard
| Challenge: | Semantic parsing maps natural language (NL) utterances into logical forms (LFs) adversarial examples are created by adding tiny perturbations to inputs but can severely deteriorate model performance. |
| Approach: | They propose to construct robustness test sets based on existing benchmark corpora and to evaluate the effect of data augmentation. |
| Outcome: | The proposed method measures the performance of the proposed parsers on robustness test sets and evaluates the effect of data augmentation. |
Deep Exhaustive Model for Nested Named Entity Recognition (D18-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a task of finding entities with specific semantic types such as Protein, Cell, and RNA in text. |
| Approach: | They propose a deep neural model for nested named entity recognition . they enumerate all possible regions or spans as potential entity mentions . |
| Outcome: | The proposed model outperforms state-of-the-art models on nested and flat NER . it achieves 77.1% and 78.4% respectively in terms of F-score, without external knowledge resources. |
Rethinking Skip Connection with Layer Normalization (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to solve the optimization problem of deep neural networks are not linear, but can be used as a modulating mechanism between the input and output. |
| Approach: | They propose to use skip connection to adjust the scale of the input and output to improve the performance. |
| Outcome: | The proposed approach improves performance and convergence of deep neural networks and can be applied to machine translation and image classification datasets. |
Psycholinguistic Tripartite Graph Network for Personality Detection (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on personality detection from online posts adopts multifarious deep neural networks to represent the posts and builds predictive models in a data-driven manner without the exploitation of psycholinguistic knowledge. |
| Approach: | They propose a psycholinguistic knowledge-based tripartite graph network, TrigNet, which consists of a tripartitic graph network and a BERT-based graph initializer. |
| Outcome: | The proposed graph network outperforms the existing state-of-the-art model by 3.47 and 2.10 points in average F1 on two datasets. |
An Effective Label Noise Model for DNN Text Classification (N19-1)
Copied to clipboard
| Challenge: | Existing methods to train deep neural networks with label noise are limited to image classification models . label noise is important because of the large number of errors and errors in training datasets . |
| Approach: | They propose a non-linear processing layer that models label noise into a convolutional neural network (CNN) they add a noise model layer on top of their target model to account for label noise . |
| Outcome: | The proposed approach is robust to label noise and can learn better sentences . it is based on extensive experiments on text classification datasets . |
ViNLI: A Vietnamese Corpus for Studies on Open-Domain Natural Language Inference (2022.coling-1)
Copied to clipboard
| Challenge: | a large-scale corpus is needed for studies on natural language inference (NLI) for Vietnamese, which can be considered a low-resource language. |
| Approach: | They propose a corpus for evaluating Vietnamese natural language inference models . they use a human-annotated corpus extracted from more than 800 online news articles . |
| Outcome: | The ViNLI corpus is created and evaluated with a strict process of quality control . the best system performance is still far from human performance (a 14.20% gap in accuracy). |
Semi-supervised Autoencoding Projective Dependency Parsing (2020.coling-main)
Copied to clipboard
| Challenge: | Existing models for semi-supervised dependency parsing use labeled data, but they require large amounts of labeles. |
| Approach: | They propose two end-to-end autoencoding models for semi-supervised graph-based projective dependency parsing. |
| Outcome: | The proposed models outperform a semi-supervised model on WSJ and UD dependency parsing data sets. |
How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection (N19-1)
Copied to clipboard
| Challenge: | Using a pre-defined vocabulary is a common approach to selecting text inputs . however, using a large vocabulary is not economical, as it limits the model's applicability on computation-or memoryconstrained scenarios. |
| Approach: | They propose a more sophisticated variational vocabulary dropout to perform vocabulary selection . they propose two new metrics to measure area under accuracy-vocab curve and Vocab Size under X% accuracy drop . |
| Outcome: | The proposed framework outperforms the baselines on the vocabulary selection problem on multiple NLP classification tasks. |
LABO: Towards Learning Optimal Label Regularization via Bi-level Optimization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for regularizing deep neural networks rely on weight decay, dropout, batch/layer normalization to converge faster and generalize. |
| Approach: | They propose a framework for training with label regularization which includes conventional LS but can also model instance-specific variants. |
| Outcome: | The proposed approach consistently yields better results than conventional regularization on seven machine translation and three image classification tasks while maintaining training efficiency. |
Word Sense Disambiguation with Knowledge-Enhanced and Local Self-Attention-based Extractive Sense Comprehension (2022.coling-1)
Copied to clipboard
| Challenge: | Word sense disambiguation (WSD) is one of the most challenging tasks in natural language processing. |
| Approach: | They propose a method to extract the right sense from a sentence context . they propose to incorporate additional examples and definitions of related senses in WordNet . |
| Outcome: | The proposed method achieves better performance than baseline models on public benchmark datasets. |
When and Who? Conversation Transition Based on Bot-Agent Symbiosis Learning Network (2020.coling-main)
Copied to clipboard
| Challenge: | a bot-agent symbiosis is a method for transparent conversation transition in online customer service applications. |
| Approach: | They propose a bot-agent symbiosis approach to solve conversation transition problems . they provide user feedback and develop deep neural networks to predict the NPS . |
| Outcome: | The proposed approach outperforms state-of-the-art methods on real-time data generated from an online service support platform. |
End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions (2023.emnlp-main)
Copied to clipboard
| Challenge: | End-to-end task-oriented dialogue (EToD) can generate responses in an end-to end fashion without modular training, which attracts escalating popularity. |
| Approach: | They present a systematic review of EToD and propose a unified perspective to summarize existing approaches and recent trends. |
| Outcome: | The proposed approaches can generate responses in an end-to-end fashion without modular training, which attracts escalating popularity. |
Modeling Multi-Granularity Hierarchical Features for Relation Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work on relation extraction focuses on constructing explicit structured features using knowledge graph and dependency tree. |
| Approach: | They propose a method to extract multi-granularity features based solely on the original input sentences. |
| Outcome: | The proposed method outperforms state-of-the-art models that even use external knowledge on three public benchmarks: SemEval 2010 Task 8, Tacred, and Tacred Revisited. |
MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in machine learning (MU) have enabled the selective removal of private or sensitive information encoded within deep neural networks. |
| Approach: | They propose to "reformulate" the task of multimodal MU in the era of MLLMs by preserving only the visual patterns associated with a given entity while preserving the corresponding textual knowledge. |
| Outcome: | The proposed method surpasses baselines that finetuned MLLMs with VQA data directly through Gradient Ascent (GA) or Negative Preference Optimization (NPO), across all evaluation dimensions. |
LayoutReader: Pre-training of Text and Layout for Reading Order Detection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for reading order detection are too laborious to annotate large datasets. |
| Approach: | They propose to use a large-scale dataset to annotate reading order information for document images . they use XML metadata to capture the reading order of WORD documents . |
| Outcome: | The proposed model performs almost perfectly in reading order detection and improves both open-source and commercial OCR engines in ordering text lines in their results. |
Automatic Annotation of Direct Speech in Written French Narratives (2023.acl-long)
Copied to clipboard
| Challenge: | a new framework for AADS annotation in written text is needed for literary studies. |
| Approach: | They propose to use automatic annotation of direct speech (AADS) in written text to compare works by different authors . they adapted a large-to-date French narrative dataset annotated with DS per word . |
| Outcome: | The proposed framework is a step further to encourage more research on the topic. |
Enhancing Language Model Hypernetworks with Restart: A Study on Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | a comprehensive investigation into optimization strategies for hypernetworks remains lacking. |
| Approach: | They propose restart optimization strategies to improve hypernetworks' performance for language models. |
| Outcome: | The proposed restart strategy improves hypernetworks' performance for language models, compared to conventional deep neural networks. |
ERASER: A Benchmark to Evaluate Rationalized NLP Models (2020.acl-main)
Copied to clipboard
Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, Byron C. Wallace
| Challenge: | State-of-the-art models in NLP are opaque in terms of how they come to make predictions. |
| Approach: | They propose to release a benchmark to measure the quality of rationales extracted by models and how faithful these rationale are to human annotators. |
| Outcome: | The proposed benchmark will enable researchers to compare models and track progress on interpretable models for NLP. |
DROWN: Towards Tighter LiRPA-based Robustness Certification (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for certifying the robustness of deep neural networks suffer from precision or scalability issues. |
| Approach: | They propose a method to certify the robustness of deep neural networks . they propose to use two pairs of linear bounds to refine pre-activation bounds . |
| Outcome: | The proposed method achieves higher certified robustness than the baseline on CNNs and 4.68 times larger certified radii than the Transformers. |
Topics to Avoid: Demoting Latent Confounds in Text Classification (D19-1)
Copied to clipboard
| Challenge: | Despite impressive performance on many text classification tasks, deep neural networks tend to learn frequent superficial patterns that are specific to the training data and do not always generalize well. |
| Approach: | They propose a method that represents latent topical confounds and a model which “unlearns” confounding features by predicting both the label of the input text and the confound. |
| Outcome: | The proposed model generalizes better and learns features indicative of the writing style rather than the content. |
Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show vulnerability of deep neural networks to adversarial examples that intentionally fool the networks. |
| Approach: | They propose a method for training a robust model to defense synonym substitution-based attacks by sampling embedding vectors for each word in an input sentence and augmenting them with the training data. |
| Outcome: | The proposed method outperforms other proposed defense methods by a significant margin across different network architectures and multiple data sets. |
Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowds (D19-1)
Copied to clipboard
| Challenge: | Incorporating Item Response Theory (IRT) into NLP tasks can provide valuable information about model performance and behavior. |
| Approach: | They propose to use IRT models generated from artificial crowds of DNNs to learn IRT. |
| Outcome: | The proposed model learning method outperforms baseline methods for two NLP tasks. |
Tensorized Embedding Layers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Using the Tensor Train decomposition, embeddings layers occupy large portion of model weights, preventing their deployment in limited resource settings. |
| Approach: | They propose a method for parameterizing embedding layers based on the Tensor Train decomposition, which allows compressing the model significantly at the cost of a negligible drop or even a slight gain in performance. |
| Outcome: | The proposed method can be plugged into any model and trained end-to-end. |
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives (D19-1)
Copied to clipboard
| Challenge: | a recent study has shown that deep neural networks are effective with various tasks . a new study examines how representations of tokens evolve between layers under different learning objectives . |
| Approach: | They use canonical correlation analysis and mutual information estimators to study how information flows across Transformer layers. |
| Outcome: | The proposed model outperforms untrained models on word identity prediction tasks . the model outpersforms models trained on other linguistic tasks based on the model's objective . |
Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings (D18-1)
Copied to clipboard
| Challenge: | Stack-Overflow, Quora, and Yahoo! Answers forums are not moderated, which results in noisy and redundant content. |
| Approach: | They use deep neural networks to learn meaningful task-specific embeddings . they incorporate the embeddables into a conditional random field model . |
| Outcome: | The proposed task improves significantly across evaluation metrics. |
Deep Weighted MaxSAT for Aspect-based Opinion Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing deep learning models lack the capability to encode explicit domain knowledge to model complex causal relationships among variables. |
| Approach: | They propose a model that uses a weighted version of MaxSAT to model logic inference . they propose to use this model to rectify erroneous predictions from deep neural networks . |
| Outcome: | The proposed model combines the benefits of high-level feature learning, knowledge reasoning, and structured learning with observable performance gain for aspect-based opinion extraction. |
RethinkCWS: Is Chinese Word Segmentation a Solved Task? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent years have seen remarkable success in the use of deep neural networks on Chinese word segmentation (CWS) however, the performance of CWS systems has gradually reached a plateau with the rapid development of deep networks. |
| Approach: | They propose a fine-grained evaluation for existing Chinese word segmentation systems that allows us to diagnose the strengths and weaknesses of existing models. |
| Outcome: | The proposed model can diagnose strengths and weaknesses of existing models and alleviate negative transfer problem when doing multi-criteria learning. |
Introducing a Large-Scale Dataset for Vietnamese POS Tagging on Conversational Texts (2020.lrec-1)
Copied to clipboard
| Challenge: | POS taggers are trained on informal texts which contain many informal inputs such as acronyms, abbreviations, out-of-vocabulary words, etc. |
| Approach: | They propose a large-scale human-labeled dataset for the Vietnamese POS tagging task on conversational texts and develop an annotation guideline to manually annotate 16.310K sentences using this guideline. |
| Outcome: | The proposed tagging scheme achieved 93.36% accuracy score and higher than the model with handcrafted features and fine-tuning BERT. |
Morfessor EM+Prune: Improved Subword Segmentation with Expectation Maximization and Pruning (2020.lrec-1)
Copied to clipboard
| Challenge: | Subword segmentation is a standard preprocessing step in many neural approaches to natural language processing. |
| Approach: | They propose to train a unigram subword model using a recursive algorithm and lexicon pruning algorithm. |
| Outcome: | The proposed method improves on the original training algorithm and improves morphological segmentation accuracy. |
Multilingual Culture-Independent Word Analogy Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | In text processing, deep neural networks use word embeddings as an input. |
| Approach: | They propose to use benchmark datasets to compare the quality of word embeddings in text processing . they use a word analogy task in Croatian, English, Estonian, Finnish, Latvian, Lithuanian, Russian, Slovenian, and Swedish . |
| Outcome: | The proposed datasets are culturally independent and cross-lingual for the languages used. |
Creating New Language and Voice Components for the Updated MaryTTS Text-to-Speech Synthesis Platform (L18-1)
Copied to clipboard
| Challenge: | a reboot of the MaryTTS system became unavoidable due to the number of people who have contributed to its development over the years. |
| Approach: | They propose a workflow to create components for the MaryTTS text-to-speech synthesis platform. |
| Outcome: | The proposed workflow is compatible with the updated MaryTTS architecture, enabling new features and state-of-the-art paradigms such as synthesis based on deep neural networks (DNNs). |
Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates (2021.acl-long)
Copied to clipboard
| Challenge: | Using negative flips, we quantify, reduce and analyze regression errors in deep neural networks. |
| Approach: | They propose to quantify, reduce and analyze regression errors in NLP models by negative flips. |
| Outcome: | The proposed model update regression has a prevalent presence across tasks in the GLUE benchmark. |
Neural Automated Essay Scoring Incorporating Handcrafted Features (2020.coling-main)
Copied to clipboard
| Challenge: | Automated essay scoring (AES) relies on handcrafted features, but recent studies have proposed a hybrid method that integrates handcrafted essay-level features into a DNN-AES model. |
| Approach: | They propose a method that integrates handcrafted features into a DNN-AES model. |
| Outcome: | The proposed method significantly improves the accuracy of existing methods. |
Word-level Textual Adversarial Attacking as Combinatorial Optimization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing word-level attack models are far from perfect because of unsuitable search space reduction methods and inefficient optimization algorithms. |
| Approach: | They propose a novel adversarial adversarialist model that incorporates word substitution and particle swarm optimization to solve two problems separately. |
| Outcome: | The proposed model achieves much higher success rates and crafts more high-quality adversarial examples as compared to baseline methods. |
UPER: Boosting Multi-Document Summarization with an Unsupervised Prompt-based Extractor (2022.coling-1)
Copied to clipboard
| Challenge: | Multi-Document Summarization (MDS) uses the extract-then-abstract paradigm, which extracts a relatively short meta-document and then feeds it into the deep neural networks to generate an abstract. |
| Approach: | They propose to use pre-trained language models to calculate document and keyword’s perplexity to boost other metrics for evaluating a document’s salience. |
| Outcome: | The proposed method can be applied as a plug-in to boost other metrics for evaluating a document’s salience, thus improving the subsequent abstract generation. |
Depth Growing for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation models with tens and even more than a hundred blocks have shown effectiveness in image recognition. |
| Approach: | They propose a two-stage approach with three specially designed components to construct deeper NMT models. |
| Outcome: | The proposed approach improves on WMT14 EnglishGerman and EnglishFrench translation tasks. |
Chinese Couplet Generation with Syntactic Information (2022.coling-1)
Copied to clipboard
| Challenge: | Chinese couplet generation aims to generate a pair of clauses with certain rules adhered . previous studies have focused on learning the correspondences between antecedent and subsequent clauses . |
| Approach: | They propose to leverage syntactic information to generate Chinese couplets by POS tags and word dependencies. |
| Outcome: | The proposed approach outperforms baselines on a Chinese couplet generation dataset. |
Uncertainty Estimation of Transformer Predictions for Misclassification Detection (2022.acl-long)
Copied to clipboard
Artem Vazhentsev, Gleb Kuzmin, Artem Shelmanov, Akim Tsvigun, Evgenii Tsymbalov, Kirill Fedyanin, Maxim Panov, Alexander Panchenko, Gleb Gusev, Mikhail Burtsev, Manvel Avetisian, Leonid Zhukov
| Challenge: | Uncertainty estimation (UE) of model predictions is crucial step for a variety of tasks such as active learning, misclassification detection, adversarial attack detection, etc. |
| Approach: | They propose to modify UE methods for Transformer models for misclassification detection in named entity recognition and text classification tasks to improve model expressiveness and computational performance. |
| Outcome: | The proposed methods outperform computationally intensive methods on misclassification detection tasks and are based on a large dataset of simulated datasets. |
Towards Accurate and Consistent Evaluation: A Dataset for Distantly-Supervised Relation Extraction (2020.coling-main)
Copied to clipboard
| Challenge: | Distant Supervision (DS) generates large-scale annotated data but has wrong labels that result in incorrect evaluation scores during testing. |
| Approach: | They build a dataset using DS-generated data as training data and hire annotators to label test data. |
| Outcome: | The proposed dataset NYTH has a much larger test set and performs more accurate and consistent evaluation. |
Local Interpretation of Transformer Based on Linear Decomposition (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on local explanation generation attempts to understand model dynamics on word-level or phraselevel by assigning importance scores on input features. |
| Approach: | They propose to interpret neural networks by linear decomposition by a Transformer model on a single input and a linear decomposing of the output to generate local explanations. |
| Outcome: | The proposed method achieves competitive performance in sentiment classification and machine translation, and fidelity of explanation. |
High Quality ELMo Embeddings for Seven Less-Resourced Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent results show that deep neural networks using contextual embeddings outperform non-contextual embedders on a majority of text classification tasks. |
| Approach: | They propose to use contextual embeddings for seven languages to train new embeddables . they also show that existing embeddibles for listed languages shall be improved . |
| Outcome: | The proposed embeddings outperform non-contextual embeddables on a majority of text classification tasks. |
Bayesian Topic Regression for Causal Inference (2021.emnlp-main)
Copied to clipboard
| Challenge: | a Bayesian topic regression model uses text and numerical information to model outcome variables. |
| Approach: | They propose a Bayesian Topic Regression model that uses both text and numerical information to model an outcome variable. |
| Outcome: | The proposed model recovers ground truth with lower bias than any benchmark model when text and numerical features are correlated. |
DARE: Towards Robust Text Explanations in Biomedical and Healthcare Applications (2023.acl-long)
Copied to clipboard
| Challenge: | Several explainability methods have been shown to be brittle in the face of adversarial perturbations of their inputs in the image and generic textual domains. |
| Approach: | They propose to adapt existing attribution robustness estimation methods to take into account domain-specific plausibility and to train networks that display robust attributions. |
| Outcome: | The proposed methods are able to characterize domain-specific plausibility and provide robust explanations on biomedical datasets. |
Effective Estimation of Deep Generative Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing techniques for parameterisation of probabilistic models by deep neural networks are difficult to use in language modelling due to posterior collapse. |
| Approach: | They propose to use variational auto-encoder to estimate probabilistic models of language by deep neural networks. |
| Outcome: | The proposed model performs reasonably well given enough resources, but a favourite can be named based on convenience. |
Unraveling Feature Extraction Mechanisms in Neural Networks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks have become indispensable across a variety of natural language processing tasks. |
| Approach: | They propose a theoretical approach based on Neural Tangent Kernels to investigate neural networks' internal mechanisms. |
| Outcome: | The proposed approach can be applied to analyze language modeling tasks . it shows that the choice of activation function can affect feature extraction . |
Recurrent Neural Networks with Mixed Hierarchical Structures and EM Algorithm for Natural Language Processing (2022.lrec-1)
Copied to clipboard
| Challenge: | A variety of hierarchical RNN models have been proposed to incorporate hierarchically-based hierarchic information in modeling languages in the literature. |
| Approach: | They propose a latent indicator layer approach to identify and learn hierarchical information and develop an EM algorithm to handle the latent indicators layer in training. |
| Outcome: | The proposed approach outperforms other RNN-based models in document classification tasks. |
Restoring ancient text using deep learning: a case study on Greek epigraphy (D19-1)
Copied to clipboard
| Challenge: | illegible parts of ancient texts must be restored by specialists, known as epigraphists, using deep neural networks to recover missing characters from text input. |
| Approach: | They propose a model that recovers missing characters from a damaged text input using deep neural networks. |
| Outcome: | The proposed model achieves a 30.1% character error rate, compared to the 57.3% of human epigraphists. |
Generalizing Backpropagation for Gradient-Based Interpretability (2023.acl-long)
Copied to clipboard
| Challenge: | Several feature-attribution methods for interpreting deep neural networks rely on computing the gradients of a model’s output with respect to its inputs, but they reveal little about the inner workings of the model itself. |
| Approach: | They propose a generalized backpropagation algorithm that generalizes the gradient computation of a model to efficiently compute other interpretable statistics about the gradient graph of neural networks. |
| Outcome: | The proposed generalized algorithm can be used to compute other interpretable statistics about the gradient graph of a neural network, such as the highest-weighted path and entropy. |
SoftRegex: Generating Regex from Natural Language Descriptions using Softened Regex Equivalence (D19-1)
Copied to clipboard
| Challenge: | Existing models that generate semantically correct regular expressions from NLs are not yet fully understood. |
| Approach: | They propose a model that rewards reinforcement learning based on the semantic equivalence between two regular expressions. |
| Outcome: | The proposed model reduces training time and produces state-of-the-art results on three benchmark datasets. |
Learning by Analogy: Diverse Questions Generation in Math Word Problem (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for solving math word problem (MWP) use shortcut learning to train solvers based on samples with a single question. |
| Approach: | They propose to generate diverse yet consistent questions from a common scenario . they then feed the equations to a question generator to obtain the diverse questions . their method leads to performance improvement on the current benchmark Math23K . |
| Outcome: | The proposed method generates diverse yet consistent questions with a variety of equations and questions . it improves on the current benchmark, which is based on the proposed method . |
Calibration Across Layers: Understanding Calibration Evolution in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated inherent calibration capabilities, where predicted probabilities align well with correctness . previous studies have linked this behavior to specific components in the final layer, such as entropy neurons and the unembedding matrix’s null space. |
| Approach: | They propose to examine how calibration evolves throughout the network's depth. |
| Outcome: | The proposed calibration direction improves calibration metrics without harming accuracy. |
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Backdoor attacks can manipulate the output of deep neural networks and possess high insidiousness. |
| Approach: | They propose a textual backdoor defense based on outlier word detection that can handle all the textual attacks. |
| Outcome: | The proposed method can handle all the textual backdoor attack situations. |
Large Corpus of Czech Parliament Plenary Hearings (2020.lrec-1)
Copied to clipboard
| Challenge: | a corpus of Czech parliament plenary sessions is a valuable resource for future research . only a few public datasets are available in the Czech language . end-to-end approaches require extensive training data to produce competitive results . |
| Approach: | They present a corpus of Czech parliament plenary sessions which is a large corpus . they combine a traditional approach with a more traditional approach . |
| Outcome: | The proposed model architectures can be used to train and evaluate speech recognition systems on a large corpus of speech data and transcripts. |
MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper (2025.emnlp-main)
Copied to clipboard
Runjia Zeng, Guangyan Sun, Qifan Wang, Tong Geng, Sohail Dianat, Xiaotian Han, Raghuveer Rao, Xueling Zhang, Cheng Han, Lifu Huang, Dongfang Liu
| Challenge: | Empirical evaluations show that Mixture of Expert Prompt Tuning outperforms state-of-the-art parameter efficient baselines on SuperGLUE. |
| Approach: | They propose a pretrain-then-fine-tune paradigm for manifold mapping using multiple prompt experts. |
| Outcome: | Empirical results show that the proposed approach outperforms state-of-the-art methods on SuperGLUE while reducing activated prompts by 79.25%. |
Selective Knowledge Distillation: Fusing LLM Semantic Strengths with DNN Efficiency for Binary Code Similarity Detection (2026.acl-long)
Copied to clipboard
| Challenge: | BinSKD is a binary code similarity detection technique that can be used in bug detection, patch analysis, and malware detection. |
| Approach: | They propose to leverage an LLM-based BCSD method as the teacher model and transfer its knowledge of high-level program semantics to various DNN-based student models. |
| Outcome: | The proposed method yields Recall@1 improvements of 14.5%–91.2% for DNN-based BCSD methods and enables HermesSim to match the teacher’s performance with orders-of-magnitude efficiency. |
Rewiring the Transformer with Depth-Wise LSTMs (2024.lrec-main)
Copied to clipboard
| Challenge: | Stacking non-linear layers allows deep neural networks to model complicated functions . but residual connections within each layer fail to fuse information from previous layers effectively . |
| Approach: | They propose a Transformer with depth-wise LSTMs connecting cascading Transformer layers and sub-layers. |
| Outcome: | The proposed model improves in English-German / French and multilingual tasks with BLEU. |
VI-OOD: A Unified Framework of Representation Learning for Textual Out-of-distribution Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is a crucial part of deep neural networks. |
| Approach: | They propose a variational inference framework which maximizes the likelihood of the joint distribution p(x, y) instead of p[y|x). |
| Outcome: | The proposed framework maximizes the likelihood of the joint distribution p(x, y) instead of p[y|x). |
Spectral Gravity Formant Estimation for Phonetic Segmentation (2026.findings-acl)
Copied to clipboard
| Challenge: | a recent study suggests that end-to-end orthographic approaches miss the mark on time . linguistic applications which require high fidelity in the temporal domain, the loss of timing information is untenable . |
| Approach: | a new algorithm uses spectral gravity to estimate formants for enhanced phonetic segmentation . a deadline-bounded expectation maximization algorithm is proposed to estimate salient speech frequencies . |
| Outcome: | a new algorithm outperforms the state-of-the-art on key clustering metrics . the proposed algorithm generates reasonable alignments across multiple languages with no a priori training. |